Image decoder and image decoding method

ABSTRACT

The present invention performs an ordinary process for reducing resolution in a case where a DCT mode is a field DCT mode. On the other hand, in a case where the DCT made is a frame DCT mode, the present invention performs inverse discrete cosine transform by changing a basis function of discrete cosine transform to another basis function, so that image data whose data of each field is thinned out is output. Due to this process, thinning out in a vertical direction which would be performed after performing full scale IDCT becomes unnecessary. Therefore, according to the present invention, it is possible to prevent occurrence of distortion which might occur when decoding image data while reducing display resolution.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to an image decoder and an imagedecoding method for obtaining image data whose size is reduced, fromimage data which has been compression-coded.

[0003] 2. Description of the Related Art

[0004] When reproducing image data which has been compressed in acompression method which supports interlaced scanning such as MPEG 2(Moving Pictures Experts Group phase 2), etc., there are some caseswhere such image data is displayed on a display device having resolutiondifferent from the size of the image data. Those cases include videodata for a HDTV (High Definition Television) displayed on an ordinarytelevision monitor, and a PC (personal computer) monitor.

[0005] Generally, in those cases, the image data is completely decoded,and then, the data size is reduced at the time the image data isdisplayed. When reducing the data size, details (high frequencycomponents) of the image are lost. Therefore, by previously cutting thehigh frequency components and then decoding the image data together withreducing the data size, reproducing performance is improved. In a casewhere image data is decoded by software on a PC, even if decoding andreproducing are performed on a low-performance CPU, load imposed on theCPU while decoding can be reduced if decoding is performed whilereducing the resolution.

[0006] “Scalable decoder without low-frequency drift” by Iwahashi,Kanbayashi, and Takaie (Technical Report of IEICE, DSP94-108, 1995.1)proposes a decoder which reduces the resolution during decoding, andthus outputs an image whose size is reduced, as described above. FIG. 6is a block diagram showing the structure of the decoder of thisproposal. When decoding compressed image data, the 4×4 IDCT (InverseDiscrete Cosine Transform) process unit 623 of this decoder reduces theresolution by performing 4×4 IDCT using only low frequency componentsamong DCT blocks. The motion compensation process unit 625 reduces thevalues of a decoded motion vector to half, and performs motioncompensation with ¼ pixel precision.

[0007] However, according to this decoding method, field information ofeach line of the image is lost by reducing the size in a verticaldirection. Thus, there arises a problem that field prediction of themotion compensation employed in MPEG 2, etc. cannot be performedproperly.

[0008] To overcome this problem, Unexamined Japanese Patent ApplicationKOKAI Publication No. 2000-59793 discloses a method of switching IDCTmethods between a field DCT mode and a frame DCT mode. According to thismethod, when the mode is switched to the frame DCT mode which appliesDCT to the top field and to the bottom field together, image data whichis obtained by performing IDCT is once divided into two fields, andfield DCT is applied to the respective fields. And by performing IDCTusing only low frequency components, size reduction is performed whilekeeping field information.

[0009] However, this method includes many calculations, thus is notsuitable for processes that require high process performance, such as adecoding process performed by software on a PC. And this method iscompletely unsuitable in a case where display resolution is reduced witha view to improving reproducing performance.

[0010] Unexamined Japanese Patent Application KOKAI Publication No.2000-175195 discloses a technique by which, when the mode is set to theframe DCT mode, IDCT is applied to all coefficients of DCT blocks, andDCT is applied to respective two pixel blocks which have been separated.However, this technique cannot solve the problem of increase incalculations, either. Further, as another example of a known art,Unexamined Japanese Patent Application KOKAI Publication No. 2000-244917discloses a technique for providing an image decoder and an imagedecoding method which simplify the decoding process using a simplestructure at the time of motion compensation. Unexamined Japanese PatentApplication KOKAI Publication No. 2000-350207 discloses a technique forapplying a transformation kernel including less calculations to an upsampling and down sampling procedures by employing generalizedorthogonal transformation. Further, Unexamined Japanese PatentApplication KOKAI Publication No. H10-191338 discloses a techniquerelating to a single coding apparatus capable of compressing an imageeffectively regardless of whether the image is a natural image or anartificial image.

SUMMARY OF THE INVENTION

[0011] The present invention was invented in view of the above problems,and it is an object of the present invention provides an image decoderand an image decoding method capable of preventing occurrence ofdistortion that may occur when decoding an image coded by an imagecoding method employing discrete cosine transform (DCT) such as MPEG 2,while reducing display resolution.

[0012] Also, the present invention provides an image decoder and animage decoding method capable of performing a process for reducingresolution (downscale decoding process) at high speed.

[0013] Thus, the present invention according to a first aspect providesan image decoder which applies a decoding process to image datacomprising:

[0014] means for applying a variable length decoding process to a firstimage data; means for applying an inverse quantizing process to a firstimage data; means for determining a mode of discrete cosine transform ofa first image data; inverse discrete cosine transform means forobtaining a second image; and

[0015] wherein a inverse discrete cosine transform means transform afirst image data into a second image having a lower degree than a firstimage data when a mode is a field mode which performs a discrete cosinetransform by each field, and transform a first image data into a secondimage with keeping field information when a mode is a frame mode whichperforms a discrete cosine transform by each frame.

[0016] The inverse discrete cosine transform in case of the frame modemay preferably be a transform process equal to a process for performingfull scale inverse discrete cosine transform in a vertical direction andthinning out image data by taking an average value of two lines arrangedat every other line with respect to each other.

[0017] Further, the average value may preferably be derived byperforming calculation of the inverse discrete cosine transform using anew basis function different from a basis function of the discretecosine transform.

[0018] The present invention according to a second aspect provides animage decoding method for obtaining image data, comprising the followingsteps: applying a variable length decoding process to image data havingfirst resolution; applying an inverse quantizing process to image data;determining a mode of discrete cosine transform of image data; obtainingimage data having second resolution having an lower degree than imagedata having first resolution when a mode is a field mode which performsa discrete cosine transform by each field; and obtaining image datahaving second resolution by applying inverse discrete cosine transformto image data which has been subjected to an inverse quantizing processwhile keeping field information when a mode is a frame mode whichperforms a discrete cosine transform by each frame.

[0019] The inverse discrete cosine transform in case of the frame modemay preferably be a transform process equal to a process for performingfull scale inverse discrete cosine transform in a vertical direction andthinning out image data by taking an average value of two lines arrangedat every other line with respect to each other.

[0020] Further, the average value may preferably be derived byperforming calculation of the inverse discrete cosine transform bychanging a first basis function which is used when the discrete cosinetransform is performed to a second basis function.

[0021] The present invention according to a second aspect provides animage decoder which applies a decoding process to image data comprising:

[0022] a first image data transformed into a second image data through avariable length decoding process, an inverse quantizing process, adiscrete cosine transform mode selection process and an inverse discretecosine transform process; and wherein a second image data transformed alower resolution data than a first image data when a discrete cosineprocess is performed by each field and, a second image data having asame resolution with a first image data when a discrete cosine processis performed by each frame.

BRIEF DESCRIPTION OF THE DRAWINGS

[0023] These objects and other objects and advantages of the presentinvention will become more apparent upon reading of the followingdetailed description and the accompanying drawings in which:

[0024]FIG. 1 is a block diagram showing the structure of an imageprocessing apparatus according to an embodiment of the presentinvention;

[0025]FIG. 2 is a diagram for explaining a frame DCT mode;

[0026]FIG. 3 is a diagram for explaining a field DCT mode;

[0027]FIG. 4 is a diagram for explaining 8 scale IDCT in the verticaldirection,

[0028]FIG. 5 is a flowchart showing the flow of a moving picturereproducing process performed by the image processing apparatusaccording to the embodiment; and

[0029]FIG. 6 is a diagram showing the structure of a conventionaldecoder which reduces resolution during decoding, and outputs an imagewhose size is reduced.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0030] We explain that an embodiment of the present invention withreference to the attached drawings. FIG. 1 is a block diagram showingthe structure of an image processing apparatus according to thisembodiment. In the image processing apparatus shown in FIG. 1, a storagedevice 1 stores an original image 11 which is compressed. An image dataprocessing device 2 decodes image data loaded from the storage device 1,and sends the decoded image data to a display device 3.

[0031] As described above, in case of applying a decoding process to acoded image while reducing resolution (i.e., applying a downscaledecoding), if inverse discrete cosine transform (IDCT) of low degree isapplied to the vertical direction of the image, field information islost. Accordingly, field prediction of motion compensation employed inthe coding method cannot be properly performed.

[0032] Therefore, the image processing apparatus of this embodimentapplies DCT of low degree while keeping field information, in a frameDCT mode employed in a coding method.

[0033] In order to achieve this, the image processing apparatus of thisembodiment shown in FIG. 1 comprises the storage device 1 which storescoded image data, the image data processing device 2 which moves inaccordance with a control of a program, and the display device 3 whichdisplays the image data.

[0034] More specifically, the storage device 1 stores the original image11 which is coded by a coding method which supports interlaced scanningsuch as MPEG 2. The image data processing device 2 comprises acompressed data buffer 21, a variable length decoding and inversequantizing process unit 22, a DCT mode selection process unit 23, afield IDCT process unit 24, a frame IDCT process unit 25, a lowresolution MC process unit 26, and a frame data buffer 27. Thecompressed data buffer 21 retains image data loaded from the storagedevice 1. The variable length decoding and inverse quantizing processunit 22 performs a variable length decoding and inverse quantizingprocess which is the first step of a decoding process. The DCT modeselection process unit 23 makes selection about a DCT mode. The fieldIDCT process unit 24 performs IDCT when the mode is set to the field DCTmode. The frame IDCT process unit 25 performs IDCT when the mode is setto the frame DCT mode. The low resolution MC process unit 26 performsmotion compensation in a state of a reduced resolution. The frame databuffer 27 stores image data to be displayed.

[0035] Image data decoded by the image data processing device 2 havingthe above described structure is displayed by the display device 3.

[0036] An operation of the image processing apparatus according to thisembodiment will now be specifically explained. As described above, thestorage device 1 of the image processing apparatus according to thisembodiment stores the original image 11 coded by a coding method whichsupports interlaced scanning such as MPEG 2. In order to decode theoriginal image 11, the image processing apparatus loads the image datato the compressed data buffer 21 first.

[0037] In this embodiment, the original image 11 is coded by a codingmethod supporting interlaced scanning, such as MPEG 2. The decodingprocess includes variable length decoding, inverse quantization, inversediscrete cosine transform, and motion compensation.

[0038] As a first step of decoding, the variable length decoding andinverse quantizing process unit 22 applies variable length decoding andinverse quantization to the data of the original image 11. The data tobe inverse-quantized has been coded by discrete cosine transform by each8×8 pixel block.

[0039]FIG. 2 schematically shows a discrete cosine transform schemeusing 8×8 pixels, This scheme is called a frame DCT mode. FIG. 3schematically shows a discrete cosine transform scheme in which DCT isperformed by rearranging lines by every other line (by rearranging linesinto fields). This scheme is called a field DCT mode. Those two modesare actually employed in MPEG2, etc.

[0040] In order to reduce resolution of the image at the time ofdecoding, inverse discrete cosine transform of lower degree than that of8×8 discrete cosine transform performed at the time of coding, isperformed. For example, in order to reduce resolution to ½ in bothvertical and horizontal directions, 4×4 inverse discrete cosinetransform is performed. However, in case of the frame DCT mode, fieldinformation is lost after 4×4 inverse discrete cosine transform.

[0041] If field information is lost, motion compensation which isperformed after inverse discrete cosine transform cannot be performedproperly, and decoding cannot be completed properly. Therefore, the DCTmode selection process unit 23 detects the DCT mode, and performsdifferent processes for the frame DCT mode and the field DCT mode. Thatis, in case of the field DCT mode, since data in each block isstructured into each corresponding field, the field IDCT process unit 24reduces the resolution by directly performing 4×4 inverse discretecosine transform.

[0042] In case of the frame DCT mode, the horizontal directionresolution is reduced to half by simply performing 4 scale IDCT.However, if 4 scale IDCT is performed to reduce the resolution in thevertical direction, field information is lost, and thereby the imagequality deteriorates. To avoid this, IDCT is performed as describedbelow.

[0043] In order to reduce the resolution in the vertical direction whilekeeping field information, the simplest way is to perform full scale (8scale) IDCT, and thin out the output data field by field.

[0044]FIG. 4 explains a method by which data obtained by performing 8scale IDCT in the vertical direction is thinned out by taking an averageof two lines arranged at every other line with respect to each other.However, this method includes many processes because it performs 8 scaleIDCT. And after IDCT, thinning out in the vertical direction isperformed. This further increases processes included.

[0045] To solve this problem, calculation indicated below is performedin the present embodiment. That is, IDCT is defined as expression 1below, in a case where output data is indicated by f(x), and DCTcoefficient is indicated by F(x). $\begin{matrix}{\left\lbrack {{EXPRESSION}\quad 1} \right\rbrack \quad {{f(x)} = {\frac{2}{N}{\sum\limits_{u = 0}^{N1}{{C(u)}{F(u)}\cos \frac{\left( {{2x} + 1} \right)u\quad \pi}{2N}}}}}} & (1)\end{matrix}$

[0046] where $\begin{matrix}{\left\lbrack {{EXPRESSION}\quad 2} \right\rbrack \quad {{C(u)} = \left\{ \begin{matrix}\frac{1}{\sqrt{2}} & \left( {u = 0} \right) \\1 & \left( {u \neq 0} \right)\end{matrix} \right.}} & (2)\end{matrix}$

[0047] An average value g(x) of two lines arranged at every other linewhich is used when thinning out the data in the vertical direction isindicated by $\begin{matrix}{\left\lbrack {{EXPRESSION}\quad 3} \right\rbrack \quad {{g(x)} = \frac{{f(x)} + {f\left( {x + 2} \right)}}{2}}} & (3)\end{matrix}$

[0048] Therefore, g(x) can be expressed as indicated below from aboveexpressions (1) and (3). $\begin{matrix}{\left\lbrack {{EXPRESSION}\quad 4} \right\rbrack \quad \begin{matrix}{{g(x)} = \quad \frac{{f(x)} + {f\left( {x + 2} \right)}}{2}} \\{= \quad {\frac{1}{2}\left\{ {{\frac{2}{N}{\sum\limits_{u = 0}^{N - 1}{{C(u)}{F(u)}\cos \frac{\left( {{2x} + 1} \right)u\quad \pi}{2N}}}} +} \right.}} \\\left. \quad {\frac{2}{N}{\sum\limits_{u = 0}^{N - 1}{{C(u)}{F(u)}\cos \frac{\left( {{2x} + 5} \right)u\quad \pi}{2N}}}} \right\} \\{= \quad {\frac{1}{N}{\sum\limits_{u = 0}^{N1}{{C(u)}{F(u)}\left\{ {{\cos \frac{\left( {{2x} + 1} \right)u\quad \pi}{2N}} + {\cos \frac{\left( {{2x} + 5} \right)u\quad \pi}{2N}}} \right\}}}}} \\{= \quad {\frac{2}{N}{\sum\limits_{u = 0}^{N - 1}{{C(u)}{F(u)}\cos \frac{\left( {{2x} + 3} \right)u\quad \pi}{2N}\cos \frac{u\quad \pi}{N}}}}}\end{matrix}} & (4)\end{matrix}$

[0049] In this case, by defining the basis function of the IDCTcalculation not as $\begin{matrix}{\left\lbrack {{EXPRESSION}\quad 5} \right\rbrack \quad {\cos \frac{\left( {{2x} + 1} \right)u\quad \pi}{2N}}} & (5)\end{matrix}$

[0050] but as $\begin{matrix}{\left\lbrack {{EXPRESSION}\quad 6} \right\rbrack \quad {{\cos \frac{\left( {{2x} + 3} \right)u\quad \pi}{2N}\cos \frac{u\quad \pi}{N}},}} & (6)\end{matrix}$

[0051] the output of the IDCT calculation automatically equals anaverage value g(x) of two lines arranged at every other line.

[0052] In a case where the IDCT calculation is performed by software,since the value of the basis function is pre-stored in a memory, thenumber of calculations is not changed due to the change of basisfunctions. Therefore, average values can be obtained with the samenumber of calculations as a normal IDCT.

[0053] Since only g(0), g(1), g(4), and g(5) are necessary as data forperforming the process of reducing the resolution, only those fourvalues need to be calculated. With those calculations, a process that isequal to performing full scale IDCT in the vertical direction andthinning out the resultant data with average values of two linesarranged at every other line can be performed quickly.

[0054] According to the present embodiment, decoding is performed byreducing resolution to half in both vertical and horizontal directionswhile keeping field information, as described above. Motion compensationis performed by the low resolution MC process unit 26 shown in FIG. 1.Specifically, motion compensation is performed with ¼ pixel precision byreducing the motion vector to half. The result of decoding is stored inthe frame data buffer 27, and then displayed by the display device 3.

[0055]FIG. 5 is a flowchart showing the flow of a moving picturereproducing process performed by the image processing apparatusaccording to this embodiment. When the decoding process is started, theoriginal image 11 is transferred to the compressed data buffer 21 in theimage processing apparatus in step S51. Then, the variable lengthdecoding and inverse quantizing process unit 22 performs variable lengthdecoding and inverse quantization (step S52).

[0056] In step S53, the DCT mode selection process unit 23 checks theDCT mode. In a case where the mode is the field DCT mode, the field IDCTprocess unit 24 performs 4×4 inverse discrete cosine transform (stepS55). Then, the flow goes to step S56.

[0057] In a case where the mode is the frame DCT mode, the frame IDCTprocess unit 25 performs inverse discrete cosine transform withcalculation of average values in step S54. Then, the flow goes to stepS56.

[0058] In step S56, the low resolution MC process unit 26 performsmotion compensation with the low (reduced) resolution which has beenobtained in the former step. Then, in step S57, the image data which hasbeen decoded completely is stored in the frame data buffer 27. Thusdecoded image data is displayed by the display device 3 in step S58.

[0059] In step S59, whether the video display operation is completed ornot is determined. In a case where it is determined that the videodisplay operation is not completed, the flow returns to step S51, anddecoding of the next image data is started. However, in a case where itis determined that the video display operation is completed, the movingpicture reproducing process is terminated.

[0060] As explained above, according to the present embodiment, in acase where the mode is the field DCT mode, a normal process for reducingthe resolution is performed. In a case where the mode is the frame DCTmode, inverse discrete cosine transform is performed by changing thebasis function of DCT, so that data of each field is output in a thinnedout state. Such calculation eliminates the need for thinning out imagedata in the vertical direction after performing full scale IDCT. Thus,number of processes included can be reduced securely.

[0061] In other words, by performing different processes in accordancewith the DCT modes to perform the process for reducing the resolutionwhile keeping interlace information, a complicated process for reducingthe resolution becomes unnecessary, and a downscale decoding processwithout deterioration of image quality can be realized. This method iseffective especially in a case where decoding is performed by software,since this method enables decoding without need for a complicatedprocess.

[0062] And by performing IDCT by changing the basis functions,calculation of average values required when thinning out image data canbe skipped. Therefore, the downscale decoding process can be performedat high speed.

[0063] According to the present invention, variable length decoding isapplied to image data having first resolution. Then, inversequantization is applied to the image data which has been variable-lengthdecoded. Thereafter, the mode of discrete cosine transform of the imagedata before being decoded (i.e., compressed image data) is determined.In accordance with the determined mode, inverse discrete cosinetransform of lower degree than that of discrete cosine transform isapplied to the image data after inverse quantization, so that image datahaving second resolution can be obtained. In a case where the mode is afield mode which performs discrete cosine transform by each field,inverse discrete cosine transform of the above-described degree isperformed. In a case where the mode is a frame mode which performsdiscrete cosine transform by each frame, inverse discrete cosinetransform is performed while keeping field information. Due to this, athinning out process in the vertical direction becomes unnecessary, anda complicated process for reducing the resolution also becomesunnecessary.

[0064] Therefore, according to the present invention, it is possible toprovide an image decoder and an image decoding method which can realizea downscale decoding process without causing deterioration of imagequality.

[0065] Further, according to the present invention, it is possible toskip calculation of average values which is required when thinning outimage data and thus to realize a downscale decoding process at highspeed, by performing calculation of inverse discrete cosine transformusing a basis function different from the basis function of discretecosine transform, that is, by performing IDCT by changing basisfunctions.

[0066] Various embodiments and changes may be made thereunto withoutdeparting from the broad spirit and scope of the invention. Theabove-described embodiment is intended to illustrate the presentinvention, not to limit the scope of the present invention. The scope ofthe present invention is shown by the attached claims rather than theembodiment. Various modifications made within the meaning of anequivalent of the claims of the invention and within the claims are tobe regarded to be in the scope of the present invention.

[0067] This application is based on Japanese Patent Application No.2001-107329 filed on Apr. 5, 2001 and including specification, claims,drawings and summary. The disclosure of the above Japanese PatentApplication is incorporated herein by reference in its entirety.

What is claimed is:
 1. An image decoder which applies a decoding processto image data comprising: means for applying a variable length decodingprocess to a first image data; means for applying an inverse quantizingprocess to said first image data; means for determining a mode ofdiscrete cosine transform of said first image data; inverse discretecosine transform means for obtaining a second image; and wherein saidinverse discrete cosine transform means transform said first image datainto said second image having a lower degree than said first image datawhen said mode is a field mode which performs said discrete cosinetransform by each field, and transform said first image data into saidsecond image with keeping field information when said mode is a framemode which performs said discrete cosine transform by each frame.
 2. Theimage decoder according to claim 1, wherein said inverse discrete cosinetransform in case of said frame mode is a transform process equal to aprocess for performing full scale inverse discrete cosine transform in avertical direction and thinning out image data by taking an averagevalue of two lines arranged at every other line with respect to eachother.
 3. The image decoder according to claim 2, wherein said averagevalue is derived by performing calculation of said inverse discretecosine transform using a second basis function different from a firstbasis function of said discrete cosine transform.
 4. The image decoderaccording to claim 3, an output of said inverse discrete cosinetransform using said second basis function is said average value of twolines arranged at every other line with respect to each other.
 5. Theimage decoder according to claim 3, a number of calculations included insaid inverse discrete cosine transform using said second basis functionis equal to a number of calculations included in inverse discrete cosinetransform before basis functions are changed.
 6. An image decodingmethod for obtaining image data, comprising the following steps:applying a variable length decoding process to image data having firstresolution; applying an inverse quantizing process to said image data;determining a mode of discrete cosine transform of said image data;obtaining image data having second resolution having an lower degreethan said image data having first resolution when said mode is a fieldmode which performs said discrete cosine transform by each field; andobtaining image data having second resolution by applying inversediscrete cosine transform to said image data which has been subjected tosaid inverse quantizing process while keeping field information whensaid mode is a frame mode which performs said discrete cosine transformby each frame.
 7. The image decoding method according to claim 6,wherein said inverse discrete cosine transform in case of said framemode is a transform process equal to a process for performing full scaleinverse discrete cosine transform in a vertical direction and thinningout image data by taking an average value of two lines arranged at everyother line with respect to each other.
 8. The image decoding methodaccording to claim 7, wherein said average value is derived byperforming calculation said inverse discrete cosine transform bychanging a first basis function which is used when said discrete cosinetransform is performed to a second basis function.
 9. The image decodingmethod according to claim 8, wherein an output of said inverse discretecosine transform using said second basis function is equal to saidaverage value of two lines arranged at every other line with respect toeach other.
 10. An image decoder which applies a decoding process toimage data comprising: a first image data transformed into a secondimage data through a variable length decoding process, an inversequantizing process, a discrete cosine transform mode selection processand an inverse discrete cosine transform process; and wherein saidsecond image data transformed a lower resolution data than said firstimage data when a discrete cosine process is performed by each fieldand, said second image data having a same resolution with said firstimage data when said discrete cosine process is performed by each frame.11. The image decoder according to claim 10, wherein said inversediscrete cosine transform process in case of said second image datahaving said same resolution with said first image data is a transformprocess equal to a process for performing full scale inverse discretecosine transform in a vertical direction and thinning out image data bytaking an average value of two lines arranged at every other line withrespect to each other.
 12. The image decoder according to claim 11,wherein said average value is derived by performing calculation of saidinverse discrete cosine transform using a second basis functiondifferent from a first basis function of said discrete cosine transform.13. The image decoder according to claim 12, an output of said inversediscrete cosine transform using said second basis function is saidaverage value of two lines arranged at every other line with respect toeach other.
 14. The image decoder according to claim 12, a number ofcalculations included in said inverse discrete cosine transform usingsaid second basis function is equal to a number of calculations includedin inverse discrete cosine transform before basis functions are changed.